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Abstract 


Throughout  the  course  of  history,  great  leaps  in  progress  and  understanding  have  been 
facilitated  through  the  questioning  of  basic  assumptions.  In  an  effort  to  uncover  critical 
opportunities  and  vulnerabilities  within  effect-based  operations  (EBO),  similar  questions 
must  be  posed  to  our  current  assumptions  underlying  EBO  tool  development  practices. 
Required  for  these  assumptions  to  be  examined  is  a  shared  understanding  of  strategy 
formulation  as  an  intensely  human  process.  The  breadth  of  approaches  used  in  recent 
years  to  help  commanders  formulate  effects-based  courses  of  action  (CoA)  is  quite 
diverse  including  expert  systems,  Bayesian  networks,  and  scenario  analysis.  All  of  these 
approaches  represent  best  guess  assumptions  of  how  to  codify  aspects  of  the  strategy 
development  process,  often  with  out  regard  for  the  principles  of  automation.  The  adverse 
unintended  consequences  made  possible  from  this  neglect  are  wide  ranging,  including  the 
potential  to  inadvertently  foster  convergent  vs.  divergent  thinking,  conditioning 
commanders  and  policy  makers  to  accept  a  dangerously  limited  view  as  an  accurate 
model  of  “real  world”  threats.  Imperative  to  avoiding  this  conceivable  eventuality  is  a 
strategic  perspective  on  EBO  tool  development  practices.  Identified  in  this  paper  are  four 
major  paradigms  or  “schools  of  thought”  of  strategic  decision  support:  autonomous, 
directive,  predictive,  and  emergent.  The  proposed  paradigms  are  offered  to  illustrate  how 
recent  EBO  tool  development  approaches  may  be  classified  and  subsequently 
characterized  based  upon  their  inherent  gravitation  to  a  particular  decision  support 
paradigm. 


TABLE  OF  CONTENTS 


Research  Imperative . 1 

Introduction . 1 

Effects  Based  Operations . 2 

Decision  Support  Paradigms . 4 

Autonomous  Paradigm . 5 

Directive  Paradigm . 6 

Predictive  Paradigm . 7 

Emergent  Paradigm . 8 

AFRL-IF  EBO  Tool  Development  Summary . 10 

Conclusion . 11 

Appendix . 12 

References . 14 


ii 


Research  Imperative 

The  Air  Force  Scientific  Advisory  Board  (SAB)  conducted  a  science  and  technology 
review  of  the  Information  Directorate’s  Rome  Research  Site  in  November  of  2003.  The 
purpose  of  this  bi-annual  panel  review  is  to  assess  the  quality  and  long  tenn  relevance  of 
research  &  development  efforts  being  conducted  by  the  USAF.  AFRL/IF  is  in  charge  of 
supporting  a  majority  of  USAF  command  and  control  efforts  including  EBO.  When 
reviewing  this  area  the  SAB  panel  developed  a  strong  belief  that: 

“IF’s  EBO  program  is  overly  focused  on  tools,  like  CAT,  without  an 
accompanying  vision  and  appreciation  for  the  underlying  decision  science 
upon  which  modern  C2  must  rest. . .  C2  is  critically  important  to  the 
warfighter,  is  a  huge  force  multiplier,  and  requires  our  very  best  scientific 
effort. . .  Although  there  is  a  lot  of  good  work  going  on  in  C2,  there  is 
concern  about  a  culture  which  appears  to  favor  tool  development  without  a 
clear  overall  plan  of  research  in  C2.” 

The  proceeding  research  is  an  effort  to  address  this  observed  deficiency  in  basic  decision 
science  research  through  general  study  of  the  dominant  paradigms  of  EBO  tool 
development.  Following  the  presentation  of  a  broad  classification  scheme  for  EBO  tool 
development  practices,  the  AFRL/IF  EBO  tool  development  practice  underway  as  of 
August  31,  2004  are  classified  in  tabular  format. 


Introduction 

In  recent  years,  a  wide  variety  of  approaches  have  been  used  to  develop  tools  that  help 
decision  makers  generate  courses  of  action  (CoA),  for  example  Bayesian  networks, 
expert  systems,  and  scenario  generation.  The  diversity  and  fundamental  differences  of 
these  approaches  has  prompted  the  need  for  a  basic  understanding  of  the  underlying 
assumptions  upon  which  the  approaches  are  based.  Although  the  approaches  differ  in 
many  aspects,  each  approach  may  be  viewed  as  pertaining  to  a  major  “school  of  thought” 
or  decision  support  paradigm.  Tool  development  approaches  thus  may  be  classified  and 
subsequently  characterized  based  upon  their  inherent  gravitation  towards  a  particular 
decision  support  paradigm.  Questioning  the  basic  assumptions  of  tool  development 
practices  is  crucial  to  avoiding  potential  adverse  unintended  consequences  that  may  occur 
as  a  result  of  the  specific  limitations  inherent  to  a  particular  approach  or  family  of 
approaches.  Decision  support  paradigms  operate  at  the  family  level,  showing  how 
approaches  may  be  aggregated  based  upon  underlying  commonalities.  This  low 
resolution  method  of  analysis  is  not  meant  to  be  comprehensive.  It  serves  only  to 
reinforce  the  focal  premise  of  this  paper,  the  need  to  question  the  basic  assumptions 
underlying  tool  development  practices  in  an  effort  to  avoid  potentially  serious  adverse 
unintended  consequences. 
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Never  before  in  history  has  the  velocity  of  information  been  so  fast,  the  speed  of  decision 
making  so  quick,  and  the  potential  for  adverse  unintended  consequences  so  great  [1].  We 
live  in  a  world  where  the  competitive  landscape  continues  to  shift  toward  hyper¬ 
competition,  where  strategic  decision  makers  are  forced  to  continuously  contend  with 
numerous  multi-criteria  decisions  that  must  be  made  with: 

•  incomplete  data 

•  uncertain  information 

•  ambiguous  goals 

•  high  stakes 

•  time  pressure 

•  lethality  of  weapons 

•  and  against  intelligent  adversaries  [2-4] 

The  characteristics  of  the  hyper-competitive  environment  of  today  represent  a  strong 
divergence  from  familiar  past  competitive  experiences.  The  cold- war  era  was  black-and- 
white,  where  the  United  States  had  a  clear  picture  of  who  were  allies,  adversaries,  and 
neutrals  [5],  Over  the  years,  intelligence  analysts  had  developed  high  fidelity  models  of 
enemy  doctrine,  capabilities,  and  even  socio-cultural  axioms.  The  relative  certainty  of 
these  adversarial  models  removed  a  great  deal  of  ambiguity  from  the  planning  process. 
Today  much  of  that  certainty  is  gone  as  our  nation  faces  new  challenges  in  the  form  of 
transnational  terrorist  threats  [6].  Once  what  was  quantifiable  or  predictable  becomes 
irrelevant.  Adversaries  no  longer  adhere  to  set,  understood,  or  even  rational  (that  is 
“western”)  rules  of  engagement.  In  addition,  they  are  constantly  evolving  their  tactics 
(i.e.  asymmetric  warfare),  adapting  in  response  to  employed  defensive  measures,  and 
construing  all  together  new  methods  of  conducting  warfare.  To  combat  an  intelligent, 
“non-rational”  adversary  requires  a  new  way  of  conducting  operations  with  increased 
precision  and  efficiency.  In  short,  there  is  a  need  for  a  basic  realignment  in  war  planning 
as  the  character  of  warfare  is  changing  and  the  degree  of  that  change  is  considerable  [7, 
8], 

Effects  Based  Operations 

Very  recently  an  emphasis  has  been  placed  on  developing  tools  to  support  the  concept  of 
“effects-based”  planning  and  operations,  where  an  “effect”  refers  to  the  full  range  of 
physical,  functional,  or  psychological  outcomes,  events,  or  consequences  resulting  from 
specific  applied  military/non-military  force  [9,  10].  Effects  come  in  an  array  of  forms 
including  direct,  indirect,  cascading,  cumulative,  collateral,  and  systemic  (for  definitions 
see  Air  Combat  Command  EBO  White  Paper  [10]).  Recent  surveys  of  the  literature  have 
observed  that  effects-based  operations  has  no  agreed  upon  definition,  but  rather  there 
exists  many  variations  and  descriptions  of  the  evolving  concept  [11,  12].  For  the  purpose 
of  this  paper,  Effects-based  operations  (EBO)  will  be  defined  as  actions  taken  against 
enemy  systems  designed  to  achieve  specific  effects  that  contribute  directly  to  desired 
outcomes  [10].  Further,  in  a  methodological  fonn  EBO  may  be  defined  as  a 
methodology  for  planning,  executing,  and  assessing  operations  to  attain  desired  strategic 
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effects  that  contribute  directly  to  military  and  political  objectives  by  shaping  the  behavior 
of  friends,  foes,  and  neutrals  in  peace,  crisis,  and  war  [5,  7,  10,  12,  13]. 


An  emphasis  on  effects  as  they  relate  to  achieving  objectives  adds  an  element  of  rigor  to 
strategy  formulation  and  execution,  requiring  decision  makers  to  articulate  why  a  set  of 
actions  will  produce  a  resulting  set  of  intended  and  unintended  effects  through  “causal 
linkages”.  Table  1  presents  a  graphical  depiction  of  causal  linkages  to  aid  in 
comprehension  of  the  concept. 


Table  1:  Graphical  Depiction  of  EBO  Methodology  [10] 


Achieving  objectives  requires  influencing  enemy  systems,  which  may  be  accomplished 
by  affecting  either  physical  or  behavioral/psychological  targets.  To  conduct  operations  in 
the  behavioral/psychological  realm  requires  an  understanding  of  opponent  adversaries, 
viewing  them  as  a  complex  adaptive  system  that  includes  soft  factors  such  as  ideologies, 
perceptions,  and  will.  The  complexity  of  these  soft  factors  solidifies  EBO  CoA 
generation  as  a  human  intensive  process.  Decision  makers  must  rely  heavily  upon  their 
training/expertise  and  support  tools  to  operate  in  the  realm  of  behavioral/psychological 
effects,  where  focus  is  placed  upon  gaining  control  of  an  enemy  rather  than  destroying  it 
through  a  war  of  attrition.  The  principle  that  the  most  advanced  form  of  warfare  is  to 
subdue  ones  enemy  with  out  fighting  is  attributed  to  Chinese  theorist  Sun  Tzu: 

“Those  skilled  in  war  subdue  the  enemy’s  army  without  battle.  They 
capture  his  cities  without  assaulting  them  and  over-throw  his  state  without 
protracted  operations”  -Sun  Tzu  [14]. 

In  an  era  of  precision  weapons,  it  makes  intuitive  sense  that  precision  operations  should 
follow.  The  recent  emphasis  on  effects-based  operations  by  senior  defense  officials  is 
due  to  its  promised  ability  to  provide  this  decisive  capability  and  the  advantage  it  affords 
strategic  decision  makers. 
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Decision  Support  Paradigms 


In  designing  technology  to  support  human  decision  makers  (commanders)  generate  and 
select  effects-based  CoAs,  a  wide  variety  of  approaches  have  been  employed  each  with 
its  own  merits  and  limitations.  All  of  these  approaches  were  based  upon  assumptions 
made  by  tool  designers  and  developers  as  to  how  the  tool  could  best  impact  CoA 
generation.  A  cause  for  concern  lies  in  the  observation  that  developers  frequently  employ 
approaches  based  upon  their  merits  with  traditionally  little  concern  for  how  the  approach 
may  compare  to  similar  approaches  to  solving  the  same  problem  [15].  This  lack  of 
reasoning  among  approaches  indicates  an  inadequate  application  of  decision  support 
systems  engineering  principles  [16],  yielding  the  potential  for  adverse  unintended 
consequences  as  an  approach  that  is  not  well  suited  for  a  particular  environment  or  task 
may  be  chosen  by  a  developer.  Supporting  the  concept  of  a  lack  of  reasoning  between 
approaches  is  the  complementary  observation  that  a  good  deal  of  the  historical 
information  on  decision  support  system  (DSS)  design  is  technically  focused  and  non¬ 
comparative  in  nature,  typically  viewing  a  single  approach  in  isolation  [17].  It  is 
imperative  that  the  efficacy  of  the  basic  assumptions  underlying  the  approaches  are 
challenged  and  compared  prior  to  technology  being  codified  and  fielded  in  support  of 
strategic  planning. 

At  the  strategic  level  the  possible  adverse  unintended  consequences  are  wide  ranging 
however  one  is  of  particular  importance  and  deserves  further  discussion.  That  is  the 
potential  to  inadvertently  foster  convergent  vs.  divergent  thinking,  conditioning 
commanders  and  policy  makers  to  accept  a  dangerously  limited  view  as  an  accurate 
model  of  “real  world”  threats,  affording  a  “false”  situational  awareness.  Effects-based 
CoA  generation  tools  may  inadvertently  lead  strategic  decision  makers  down  a  narrow 
path,  creating  “blind  spots”  in  the  organizations  perception  of  environmental  threats  [18]. 
Imperative  to  avoiding  this  conceivable  eventuality  is  a  strategic  perspective  on  EBO  tool 
development  practices.  This  implies  a  great  responsibility  for  the  development 
community  to  understand  how  humans  formulate  courses  of  action  and  consider 
alternatives  in  the  face  of  adverse  uncertainty.  Tool  developers  must  use  this  knowledge 
as  a  critical  lens  to  further  question  their  underlying  assumptions  and  justification  for 
selecting  a  particular  approach  from  among  competing  alternatives. 

Specifically  identified  in  this  paper  are  four  major  paradigms  or  “schools  of  thought”  of 
strategic  decision  support:  autonomous,  directive,  predictive,  and  emergent.  The 
proposed  paradigms  are  offered  to  illustrate  how  recent  EBO  tool  development 
approaches  may  be  classified  and  subsequently  characterized  based  upon  their  inherent 
gravitation  to  a  particular  decision  support  paradigm.  This  classification  is  not  meant  to 
be  comprehensively  exhaustive.  They  are  presented  to  facilitate  discussion  on  the  need 
for  EBO  tool  development  to  properly  address  improving  decision  making  capabilities  in 
the  “real  world”.  This  paper  denotes  the  distinctions  between  the  various  decision 
support  paradigms  through  a  conceptual  discussion  of  their  defining  aspects  and 
implications  on  technology  design. 


4 


Autonomous  Paradigm 


One  of  the  easily  understood  paradigms  of  decision  support  is  the  autonomous  paradigm 
which  is  characterized  by  machine  autonomy  from  the  human  decision  maker.  The 
autonomous  paradigm  can  be  represented  by  fully  automated  decision  support  systems: 
for  example,  real  time  software  that  matches  mission  objectives  and  available  resources 
to  prioritize  targets. 

The  central  idea  of  the  autonomous  paradigm  is  the  belief  that  a  certain  task  or  set  of 
tasks  can  be  fully  automated,  thus  removing  the  human  from  the  decision  process.  This 
is  generally  accomplished  by  developing  a  set  of  rules  that  would  be  applied  to  the  data 
being  processed  by  the  decision  support  system.  The  developed  rules  are  assumed  to  be 
part  of  a  closed  loop  system,  meaning  that  there  are  typically  very  few  exceptions  to  the 
rules,  allowing  for  the  system  to  run  for  the  most  part  autonomous  of  human  control. 

A  foundational  insight  made  by  Simon  in  the  field  of  decision  science  is  that  decision 
problems  may  be  seen  to  exist  on  a  continuum  from  programmed  routine,  repetitive, 
well-structured,  easily  solved  to  the  polar  opposite  end  of  the  spectrum  being  non- 
programmed,  new,  novel,  ill-structured,  difficult  to  solve  [19].  If  viewed  in  this  manner, 
tasks  that  fall  under  the  autonomous  paradigm  would  generally  be  located  at  the  lower 
end  of  the  decision  continuum,  representing  the  very  well  structured,  routine,  repetitive 
tasks  that  are  easily  formulated  and  optimized.  Typically  autonomous  systems  operate  in 
relatively  stable  environments  as  exceptions  to  the  predefined  rules  would  require  human 
intervention. 

The  advantages  of  developing  a  DSS  under  the  autonomous  paradigm  are 
straightforward.  It  moves  the  task  from  human  to  machine,  eliminating  human  error  and 
allowing  the  decision  maker  to  focus  on  more  pertinent,  strategic,  higher  level  decision 
issues.  Certainly  any  task  that  is  amenable  to  being  fully  automated  in  this  fashion 
should  be  due  to  the  obvious  advantages,  but  is  it  possible  that  technologists  have  in  the 
past  and  even  today  are  continuing  to  develop  autonomous  tools  for  tasks  or 
environments  for  which  they  are  ill-suited?  The  limitations  of  autonomous  systems  are 
equally  as  clear  as  their  advantages;  exceptions  to  predefined  rules  can  cause  disastrous 
results,  with  the  severity  of  the  mistake  being  dependent  upon  what  information  is 
overlooked,  discarded,  misinterpreted,  or  otherwise  mishandled. 

So  why  then  if  the  limitations  of  such  systems  are  so  clear  would  a  DSS  developer 
attempt  to  fit  a  decision  task  under  the  autonomous  paradigm  if  it  really  would  be  better 
off  being  developed  under  a  different  paradigm?  The  answer  can  be  partially  found  by 
studying  the  law  of  the  hammer,  which  states  that  if  you  give  a  child  a  hammer  he  or  she 
will  use  it  on  everything  encountered  [15].  Hopple  goes  on  to  explain  that: 

"often  individuals  who  devote  their  professional  careers  to  learning  and 
applying  decision  analysis,  artificial  intelligence,  or  some  other  domain 
will  be  understandably  reluctant  to  admit  that  tools  and  methods  outside 
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their  domain  may  be  equally  or  even  more  appropriate  to  solve  a  given 
problem”  [15]. 

It  is  possible  that  the  limitations  of  the  paradigm  are  overlooked  or  trivialized  due  to 
narrow,  specialized  background  and  nature  of  the  technological  developers.  A  limited 
focus  mentality  is  typical  for  a  human  specialist,  however  it  represents  a  significant 
danger  to  developing  effective  strategic  decision  support  tools  in  an  era  when  rarely  any 
single  individual  or  community  of  expertise  will  be  capable  of  dealing  with  the 
complexity  of  current  military  operations  [20]. 

When  developing  tools  under  the  autonomous  paradigm  it  is  important  for  developers  to 
understand  that  full  automation  does  not  simply  supplant  human  activity  but  rather 
changes  it,  often  in  ways  unintended  and  unanticipated  by  the  designers  of  the  automation 
[21].  To  gain  a  better  understanding  of  the  assertion  of  possible  unintended 
consequences  prior  to  operational  fielding  requires  technologists  to  conduct  testing 
(evaluation  assessments)  on  the  tools  in  simulated  environmental  contexts  similar  to  the 
conditions  that  the  tools  will  face  once  fielded  [1,  22]. 


Directive  Paradigm 

The  directive  paradigm  of  decision  support  is  characterized  by  machine  centered 
guidance  of  either  human  decision  makers  or  other  technological  components.  This 
paradigm  may  be  represented  by  limited  interaction,  machine  centered  decision  support 
systems:  for  example,  case  based  reasoning  techniques  and  other  forms  of  prototypical 
expert  systems. 

The  central  focus  of  the  directive  paradigm  is  the  formulation  of  procedural  knowledge 
(step  by  step  instructions)  based  upon  available  descriptive  knowledge  (data  or 
information)  that  has  been  collected  regarding  the  current  decision  making  situation. 
The  prescriptive  direction  may  be  used  to  orchestrate  both  human  and  machine  behavior, 
potentially  circumventing  the  human  decision  maker  entirely  by  directing  other 
technological  components  to  accomplish  the  task  at  hand.  Human  interaction  with  a 
directive  DSS  thus  may  range  from  moderate  to  non-existent.  The  human  decision  maker 
might  be  called  upon  to  input  certain  values,  make  simple  choices,  and  conduct  limited 
analysis  but  as  in  the  autonomous  paradigm,  the  principle  decision  maker  will  remain  the 
pre-programmed  machine  (i.e.  software...,  which  essentially  is  the  developer  acting  as 
the  situational  commander). 

The  advantages  of  DSS  tools  under  the  directive  paradigm  have  traditionally  been  linked 
to  knowledge  engineering.  Directive  tools  aim  to  formalize  procedural  knowledge.  The 
advantages  of  such  codified  knowledge  are  clear,  including  increasing  the  availability, 
understandability,  and  survivability  of  the  knowledge  or  infonnation.  In  semi-structured 
environments  where  there  exists  a  predefined  decision  space  and  a  definite,  regular, 
timely  information  stream  from  the  external  environment  the  directive  paradigm  would 
work  reasonably  well. 
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The  issue  of  the  potential  oversimplification  of  complex  environments  must  be  addressed. 
Rarely  are  strategic  problem  spaces  simplistic  enough  that  complete  or  near-complete 
information  may  be  obtained,  allowing  for  a  linear  progression  from  problem  design  to 
choice  [23].  This  may  be  the  result  of  tool  designers  employing  an  overly  analytical  view 
of  human  decision  processes.  For  example,  Mintzberg  described  the  process  of  decision 
making  in  tenns  of  three  major  components: 

•  problem  identification 

•  development  of  alternative  solutions 

•  and  selection  among  the  alternatives  [24], 

Such  a  partitioned  view  of  decision  making  appears  well  suited  for  mathematical 
treatment  yielding  the  false  perception  that  the  world  may  be  reduced  to  simple  formulas 
with  an  optimum  choice  for  each  decision  [25].  However,  recent  studies  have  shown  that 
expert  decision  makers  approach  and  solve  problems  in  radically  different  ways  than  such 
a  simplistic  framework  would  lead  one  to  believe  [3,  4].  Naturalistic  decision  making 
researchers  believe  that  expert  commanders  make  use  of  many  cognitive  capabilities  on  a 
daily  basis  that  are  non-transferable  (e.g.  non-codeable)  such  as  intuition  and  mental 
simulation  [3].  The  non- transferability  of  these  human  capabilities  creates  limitations  for 
the  directive  paradigm  and  also  has  resounding  consequence  for  the  need  to  encourage 
the  training,  development,  and  validation  of  expert  decision  makers. 

When  developing  technology  where  the  machine  will  be  the  principle  decision  maker  it  is 
important  to  stringently  examine  what  level  of  “intelligence”  the  proposed  system  will 
possess.  The  results  of  this  examination  would  be  to  determine  if  the  level  of  intelligence 
is  properly  suited  for  the  task  at  hand,  noting  specifically  what  the  system  is  and  is  not 
capable  of  handling.  The  examination  should  also  attempt  to  show  how  the  system  will 
affect  the  mental  decision  processes  of  the  human  decision  maker.  For  example,  will  the 
human  trust  the  results  of  the  system,  how  adaptable  will  the  system  be  to  an  individual 
decision  maker,  and  most  importantly,  in  what  capacity  does  the  system  improve  the 
decision  capability  of  a  commander  or  strategic  command  team? 


Predictive  Paradigm 

The  predictive  paradigm  of  decision  support  is  characterized  by  the  use  of  competitive 
intelligence  data  and  other  forms  of  environmental  evidence  to  assess  predictive, 
probabilistic  estimates  of  event  likelihood.  The  predictive  paradigm  can  be  represented 
by  “white  board”  decision  support  tools  that  ask  strategic  decision  makers  to  layout 
courses  of  action  and  then  assess  the  probabilities  for  each  outcome,  event,  or 
consequence  pertaining  to  the  CoA.  Example  approaches  include  Bayesian  and  Colored 
Petri  Nets  to  understand  event  influence  and  timing. 

The  predictive  paradigm  derives  its  validity  from  the  axiom  that  many  events  in  the  world 
are  dependent  upon  one  another.  This  event  dependency  assumes  that  results  of  past  or 
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planned  future  events  may  be  used  to  predict  future  outcomes,  events,  or  consequences 
following  the  events  occurrence.  For  example,  historically  an  enemy  may  have  followed 
a  set  doctrine,  thus  the  adversaries  response  to  a  planned  action  (within  this  past  doctrine) 
may  be  predicted.  As  was  the  case  during  the  cold  war,  where  both  the  U.S.  and  the 
U.S.S.R.  spent  years  understanding  and  fine  tuning  an  accurate  model  of  each  others 
doctrinal  guidelines.  Beyond  extrapolating  the  past,  event  dependency  also  allows 
decision  makers  to  generate  inferences  regarding  future  events.  For  example,  if  actions  a 
&  b  are  taken  against  enemy  x,  enemy  x  will  most  likely  respond  by  doing  y.  The  most 
likely  indicates  the  assessed  probability  of  future  state  Y  occurring. 

Tools  developed  under  the  predictive  paradigm  have  achieved  a  measure  of  success  in 
recent  experiments  but  also  a  measure  of  criticism.  In  an  effort  to  stave  off  criticism 
from  focusing  on  any  one  tool  in  particular,  which  is  not  the  purpose  of  this  paper  but 
rather  to  challenge  designers  to  question  the  basic  assumptions  of  their  approaches,  the 
names  of  the  particular  tools  will  be  omitted.  In  stating  that,  recently  developed 
predictive  tools  have  been  observed  to  have  benefits  in  terms  of  increasing  the  level  of 
analysis  and  reasoning  commanders  must  perform  over  a  set  of  potential  courses  of 
action.  However  predictive  tools  require  decision  makers  to  engage  in  probability 
estimation,  a  skill  which  humans  have  consistently  performed  poorly  at.  Many 
researchers  have  advised  against  attempting  to  assign  probabilities  to  events  or  trends. 
Attempting  to  define  probabilities  is  difficult.  To  do  so  will  cause  the  planning  team  to 
dissipate  a  lot  of  energy  for  no  real  advantage  [26]. 

In  addition,  if  a  decision  maker  trustfully  employs  a  probabilistic  tool;  it  signifies  that  he 
or  she  believes  that  the  numeric  estimates  being  assigned  to  future  events  bear  a 
somewhat  direct  and  meaningful  relationship  to  the  eventual  outcomes  of  those  events. 
Essentially  the  decision  maker  believes  the  probabilistic  model  accurately  portrays  the 
future  environment  with  some  degree  of  certainty.  Thus  to  improve  the  model  is  to 
improve  ones  understanding  of  the  future  environment.  However  this  may 
unintentionally  change  the  task  of  strategy  development  to  being  a  matter  of 
understanding  the  future  by  labeling  it  [27],  rather  than  a  matter  of  continually  re¬ 
evaluating  the  evolving  situational  environment. 

When  developing  technology  under  this  paradigm  it  is  important  for  developers  to 
understand  the  needs  of  a  strategic  decision  maker  as  well  as  the  difficulty  of  prediction 
in  a  complex  environment.  Predictive  tools  must  convey  an  appreciation  of 
understanding  all  possible  future  states  to  the  decision  maker,  otherwise  constructed 
predictions  will  yield  a  dangerously  bounded  perspective  of  events  to  come,  overlooking 
new  opportunities  and  risks  [26,  28]. 


Emergent  Paradigm 

Within  the  emergent  paradigm,  the  goal  of  technology  is  to  help  the  human  strategic 
decision  maker  emerge  with  a  greater  understanding  of  how  the  future  competitive 
landscape  will  take  shape  in  the  terms  of  potential  future  scenarios.  At  the  heart  of  the 


8 


emergent  paradigm  is  the  belief  that  at  a  strategic  level,  the  future  is  wildly  unpredictable 
and  can  not  be  treated  in  a  logical,  predefined  formulation.  In  a  classic  article,  Simon 
noted  that .  .  . 

“many,  perhaps  most,  of  the  problems  that  have  to  be  handled  at  middle 
and  high  levels  in  management  have  not  been  made  amenable  to 
mathematical  treatment,  and  probably  never  will”  [19]. 

The  emergent  paradigm  by  nature  is  meant  to  improve  the  strategic  decision  maker’s 
appreciation  for  the  vast  uncertainty  surrounding  future  events.  The  paradigm  focuses 
tool  development  efforts  on  aiding  the  human  expert  to  observe  and  reflect  rather  than  to 
formulate  and  estimate.  Example  approaches  are  based  upon  scenario  generation, 
planning,  and  analysis. 

To  address  the  unpredictable  nature  of  future  environments,  both  researchers  and 
practitioners  have  employed  scenario  planning  since  the  1970’s  [29].  We  will  use  the 
Brauers  and  Weber  scenario  definition:  a  description  of  a  possible  future  state  of  an 
organization’s  environment  considering  possible  developments  of  relevant 
interdependent  factors  in  this  environment  [28].  Scenarios  are  a  natural  fit  for  the 
emergent  paradigm.  Their  development  encourages  an  exhaustive,  creative  look  at  future 
possibilities  and  encourages  the  discovery,  inclusion,  and  consideration  of  outlier  events 
(unlikely  or  seemingly  unfeasible  events)  in  the  strategic  planning  process.  The  strategic 
use  of  scenarios  has  been  formalized  in  numerous  methodologies  both  analytical  and 
experiential.  Central  to  both  is  the  need  for  strategists  to  run  through  each  possible 
uncertain  future,  allowing  them  to  work  through  the  importance,  interrelationships,  and 
consequences  of  the  uncertainties  involved  [30]. 

The  emergent  paradigm  has  numerous  advantages  when  applied  to  the  strategic  level  of 
conflict.  Tools  developed  under  this  “school  of  thought”  aid  the  human  decision  maker 
to  develop  situational  awareness,  consider  the  many  possible  ways  a  situation  may 
evolve,  and  ultimately  formulate  a  dynamic,  response  strategy  that  incorporates  an 
appreciation  for  the  breadth  of  possible  future  scenarios.  Typically  this  strategy 
development  process  involves  first  generating  a  comprehensive  set  of  future  scenarios  by 
varying  the  potential  actions/fluctuations/responses  of  situational  military,  economic, 
political,  and  environmental  actors,  then  monitoring  the  evolving  adversarial  environment 
to  detennine  which  of  the  developed  potential  future  scenarios  most  closely  approximates 
the  current  environmental  situation.  In  this  way  the  emergent  paradigms  focus  is  placed 
upon  understanding  and  monitoring  the  future  as  an  evolving  process,  rather  than  on 
forecasting  the  future  based  upon  probabilistic  predictions  and  corresponding 
assessments. 

It  is  important  to  note  that  there  have  been  many  analytic  approaches  to  model  the 
scenario  analysis  process  mathematically,  such  as  cross-impact  analysis,  the  Brauers  and 
Weber  method,  and  fuzzy  scenario  analysis  [28,  31,  32].  But  it  is  principal  to  recognize 
that  such  methods  represent  a  divergence  from  the  major  “school  of  thought”  of  the 
emergent  paradigm.  Granted  that  adding  an  analytical  component  to  the  process  can  be 
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beneficial,  it  however  focuses  the  intention  of  the  strategic  decision  maker  inward 
towards  manipulating  identified  scenarios  rather  than  outward,  towards  the  ever  evolving 
adversarial  environment.  These  analytical,  typically  probabilistic  methods  will  possess 
many  of  the  same  characteristics  of  approaches  encountered  in  the  predictive  paradigm. 

Scenario  planning  has  several  limitations  as  a  tool  of  the  emergent  paradigm.  Depending 
on  the  breadth  and  depth  of  the  scenarios,  their  development  may  be  costly  in  tenns  of 
time,  energy,  and  resources.  Another  issue  arises  when  environmental  uncertainty 
increases  to  the  point  where  the  number  of  unknown  unknowns  greatly  exceeds  the 
number  of  known  unknowns  [33].  The  primary  role  of  a  strategic  tool  in  such  an 
incomprehensible  uncertain  environment  would  be  to  aid  in  transforming  unknown 
unknowns  into  known  unknowns,  a  task  focused  less  on  planning  and  more  on 
intelligence,  surveillance,  reconnaissance  efforts. 


AFRL-IF  EBO  Tool  Development  Summary 

Applying  the  low-resolution  decision  support  paradigm  framework  articulated  in  this 
paper  to  the  AFRL/IF  EBO  Tool  list  produced  the  following  summary  classification  chart 
(see  appendix  for  tool  descriptions). 

AFRL/IF  EBO  Tools  by  Predominant  Paradigm* 


Predictive  Paradigm 

Causal  Analysis  Tool 

Fusion  for  EBO 

Holistic  C4I  Human  Factors  Modeling  Tool 
Effects-based  Wargaming  Simulation 

Adv.  Modeling  for  Dynamic  CoA  analysis 
Dynamic  CoA  Development 

Directive  Paradigm 

Autonomous  Paradigm 

Emergent  Paradigm 

Center  of  Gravity  Analysis  Tool 

Strategy  Development  Tool  (loosely) 

Target  Systems  Analysis 

Effector  (loosely) 

Athena 

*Tools  not  far  enough  along  in  development  to  be  classified  include  the  Complex  Adaptive 
Systems  (CAS)  and  Air  Operations  Center  Process  Assessment  Tool  (AOC  /  PAT). 

The  most  striking  aspect  of  this  diagram  is  the  overwhelming  number  of  tools 
being  developed  under  the  predictive  paradigm  by  AFRL/IF.  Tool  development 
practices  within  the  organization  have  undoubtedly  favored  a  predictive  posture 
with  an  anon  AFRL/IF  developer  stating  first-hand  “If  you  don’t  predict,  what 
else  would  you  do?”  No  reviewed  tools  fit  clearly  under  the  directive  paradigm. 
The  tools  listed  under  the  autonomous  paradigm  (e.g.  COG-A,  TSA,  &  Athena) 
are  all  only  partially  autonomous  tools  but  the  school  of  thought  that  developed 
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them  was  of  an  automated  origin,  thus  in  this  low-resolution  framework  they  have 
been  placed  in  the  autonomous  paradigm. 

The  Strategy  Development  Tool  (SDT)  developed  by  Alphatech  and  it’s  sister 
tool  Effector  developed  by  ISX  have  been  listed  under  the  emergent  paradigm 
because  the  tools  are  human  centric,  however  the  method  of  decision  support  is 
only  loosely  emergent.  SDT  and  Effector  both  simply  provide  a  “white-board” 
tool  that  commanders  may  use  to  articulate  potential  courses  of  action  within  an 
effects-based  taxonomy  and  framework.  These  tools  do  not  help  commanders 
consider  the  breadth  of  possible  future  that  may  occur  or  in  any  way  foster  an 
emergent  thought  process. 


Conclusion 

Effects-based  operations  is  currently  being  supported  by  a  variety  of  tools  which  have 
been  developed  under  all  four  of  the  preceding  paradigms.  It  is  imperative  that 
technology  designers  be  aware  that  each  paradigm  has  its  own  inherent  advantages  and 
limitations.  The  insight  gained  from  questioning  the  basic  assumptions  of  tool 
development  approaches  may  help  developers  avoid  the  adverse  unintended 
consequences  made  possible  from  potential  “blind  spots”  that  the  tool  may  create  in  the 
decision  maker’s  perception  of  environmental  threats.  A  true  cause  for  concern  may  be 
that  the  irrationality  of  terrorist  actions  actually  increases  their  chances  of  inadvertently 
successfully  targeting  a  “blind  spot”.  For  example,  as  enemies  of  state  and  country  no 
longer  have  a  homeland  at  risk  and  thus  may  be  willing  to  chance  using  weapons  of  mass 
effect  indiscriminately.  In  such  an  environment  decision  makers  will  face 
hypercompetitive  pressure  that  will  require  the  decision  technology  designed  to  support 
their  strategy  formulation  be  meticulously  analyzed,  questioned,  tested,  and  evaluated. 

In  the  process  of  information  age  transformation,  experimentation  and  evaluation  will 
become  the  foundational  pillars  upon  which  further  understanding  of  decision  support 
paradigms  will  rest.  To  further  understand  specifically  how  a  particular  technological 
concept,  approach,  or  technology  impacts  human  and  organizational  strategic  decision 
processes  will  require  a  continuous  experimentation  campaign,  or  series  of  experiments 
that  test  and  evaluate  the  efficacy  of  the  object  of  interest  [9,  20],  These  evaluative 
experiments  will  also  serve  as  the  primary  vehicle  to  analyze  the  underlying  assumptions 
of  current  EBO  tool  development  practices.  Any  attempt  to  codify  aspects  of  the 
strategic  effects-based  CoA  generation  process  will  inherently  structure  and  affect  human 
decision  making  thought  processes  and  activity,  thus  all  effects-based  strategy 
formulation  tools  (and  arguably  all  tools  in  general)  must  be  rigorously  tested  and 
evaluated  to  detennine  the  tools  impact  on  human  cognitive  processes.  The  decision 
support  paradigms  articulated  in  this  paper  are  offered  to  provide  an  aggregated  vantage 
point  upon  which  the  basic  assumptions  of  current  EBO  tool  development  practices  may 
be  viewed  and  questioned.  The  revision  and  expansion  of  this  vantage  point  is  inevitable 
and  welcomed  in  light  of  future  testing  and  evaluation. 
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Appendix 


The  Air  Force  Research  Laboratories  Information  Directorate  (AFRL/IF)  has  been 
addressing  in  recent  years  the  need  to  develop  technologies  in  support  of  effects-based 
operations.  To  date  AFRL/IF  has  two  core  technologies  under  its  EBO  development 
thrust  being  the  Causal  Analysis  Tool  (CAT)  and  the  Strategy  Development  Tool  (SDT), 
however  several  other  efforts  are  at  various  states  in  the  proverbial  development 
“pipeline”  and  a  few  efforts  have  met  with  limited  results  and  thus  have  been  terminated. 
Efforts  that  are  still  in  the  development  “pipeline”  include  Fusion  for  EBO  (FEBO), 
Target  System  Analysis  (TSA),  Center  of  Gravity  Analysis  Tool  (COG-A),  and  the 
Holistic  C4I-Human  Factors  Modeling  Tool.  The  only  terminated  effort  uncovered  was 
the  Effects-based  Operations  Wargaming  Simulation  (EBOWS). 

AFRL/IF  EBO  Tools  Development  Efforts  List 

Causal  Analysis  /  Operational  Assessment  Tool  (CAT/OAT)  -  Predicts  the  probability 
of  achieving  a  commander’s  intent  for  a  blue  CoA.  CAT  helps  commanders  reason  over 
cause/effects  relations  for  a  given  campaign  over  time,  providing  for  tradeoff  analysis 
and  drill  down  capability.  CAT  provides  the  ability  to  perform  operational  assessment 
based  upon  fused  BDA/PBA  evidence. 

Strategy  Development  Tool  (SDT)  -  Supports  development  of  an  effects  based  CoA. 
SDT  decomposes  commander’s  intent  into  hierarchies  of  desired  effects,  tasks,  and 
causal  linkages.  SDT  forces  commanders  to  structure  and  articulate  their  plans  in  an 
effects  based  taxonomy  and  process  framework. 

Fusion  for  EBO  (FEBO)  -  The  tool  is  used  to  detennine  if  measures  of  effectiveness 
(MOEs)  and  indicators  from  SDT  are  being  met  as  well  as  to  provide  accrued  and  fused 
multi-INT  evidence  to  SDT/CAT  using  the  AFRL  Advanced  Sensor  Fusion  Architecture. 

Center  of  Gravity  Analysis  Tool  (COG-A)  -  COG-A  represents  a  semantic  information 
integration  effort  to  create  a  single  ontology-based  model  that  provides  a  “system  of 
systems”  view  of  adversary  capabilities.  COG-A  provides  “dependencies  linkage” 
reasoning  to  support  effects-based  COG  analysis  as  well  as  automated  integration  of 
information  from  multiple  data  sources  into  the  single  ontology-based  model. 

Holistic  C4I-Human  Factors  Modeling  Tool  -  Effort  aims  to  develop  an  architecture  to 
posture  CAT’s  Dynamic  Bayesian  Networks  behavioral  models  and  responses  to 
proposed  psychological  or  influence  operations  for  independent  analysis  of  for  reuse  by 
NASIC’s  C2  process  models. 

Effects-based  Operations  Wargaming  Simulation  (EBOWS)  -  Analytic  tool  for  CoA 
comparison  based  on  engagement-level,  attrition-based  wargaming. 

Target  Systems  Analysis  (TSA)  -  Tool  allows  users  to  develop  candidate  target  lists 
based  upon  given  effects-based  mission  objectives. 
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Athena  -  Tool  provides  a  TSA  and  COG  analysis  capability  that  integrates  the  Warden, 
Barlow,  and  PMESII  models  and  focuses  on  doctrinally  correct  TSA  for  Joint  Pub  3-60. 

Effector  -  Tool  used  to  build  effects-based  CoAs.  Effector  is  ISX’s  EBO  ontology 
strategy  development  tool  or  SDT.  SDT  was  developed  by  Alphatech. 

Dynamic  Course  of  Action  Development  (DCOAD)  -  Tool  designed  to  predict  and 
asses  the  impact  of  emerging  threats  against  plans  in  development  by  continuously 
monitoring  the  battlespace  for  emerging  targets.  The  tool  also  updates  Co  A  assessment. 

Complex  Adaptive  Systems  (CAS)  -  Two  phase  1  efforts  looking  at  how  CAS  theory 
may  apply  to  effects  based  operations  and  CoA  generation. 

Adversary  Modeling  for  Dynamic  Course  of  Action  Analysis  -  Development  of 
techniques  for  generating  hypothesis  of  adversary  actions  and  identify  the  associated  goal 
or  rational.  The  tool  would  utilize  Bayesian  Networks  to  capture  adversary 
computational  model  for  actions  and  rational. 

Air  Operations  Center  Process  Assessment  Tool  (AOC  PAT)  -  Executable  model  of 
the  AOC  for  the  analysis  of  process  improvement. 
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